Deep Networks with Internal Selective Attention through Feedback Connections

摘要

Traditional convolutional neural networks (CNN) are stationary andfeedforward. They neither change their parameters during evaluation nor usefeedback from higher to lower layers. Real brains, however, do. So does ourDeep Attention Selective Network (dasNet) architecture. DasNets feedbackstructure can dynamically alter its convolutional filter sensitivities duringclassification. It harnesses the power of sequential processing to improveclassification performance, by allowing the network to iteratively focus itsinternal attention on some of its convolutional filters. Feedback is trainedthrough direct policy search in a huge million-dimensional parameter space,through scalable natural evolution strategies (SNES). On the CIFAR-10 andCIFAR-100 datasets, dasNet outperforms the previous state-of-the-art model.